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The statistician has n independent estimates of a 
parameter he knows is positive^ but^ as is the case in 
components-of -variance problems, some of the estimates may be 
negative* If the n estimates^are to be combined into a single number, 
we compare the obvious rule, that of averaging the n values and 
taking the positive part of the result, with that of averaging the 
positive parts* Although the estimator generated by the second rule 
is not consistent, it is shown by numerical calculation that for 
small h it has a smaller mean square error than the first over a 
considerable region of the parameter space, and that for n = 2 or 3 
the second is mihimax relative to the first over a region consisting 
of almost the whole parameter space. The distribution of each of the 
n estimates is assumed to be either Gaussian or the distribution of a 
weighted difference of two independent chi-squares iid.th known degrees 
of freedom, as in one-way components of variance. Some other simply 
calculated estimators, including the positive part of the median, are 
studied for the chi-square difference case with (2,2) dSgrees of 
freedom and n =^ 3. (Author) 
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COMBINING UNBIASED ESTM^TES OP A PARAIffiTER MOim TO BF POSITPvE 

T* W. F. Strou(f 

ABSTRACT 

The statistician has n independent estimates of a parametex* he knows . 
is positive but^ as is the case in components -of -variance problems, some of 
the estimates may be negative. If the n estimates are to be combined into 
a single number, v/e compare the obvious rule, that 'of averaging the n 
values and taking the positive part of the result, v;ith that of avei^aging 
the positive parts. Although the estim.ator generated by the second rule is 
not consistent, it is shov;n by numerical calculation that for small n it 
has a smaller mean square eri^or than the first over a considerable region 
of _the pai'ameter space, and that for n =^ 2 or 5 the second is minimax 
relative to the first over a 'region consisting of almost the v.'hole parameter 
space. The distribution of each of the n estimates is assumed to be 
either Gaussian or the distribution of a v/eighted difference of t\:o indepen- 
dent chi-squares i:ith knovm degrees of freedom, as in one-v/ay components of 
variance. Some other simply calculated estimators, including the positive 
part of the median, are studied for the chi -square difference case with 
(2,2) degrees of freedom and n = 5 • 



COMBINING UNBIASED ESTBIATES OF A PARAMETER MOW TO BE POSITr/E 

I. INTRODUCTION 

Sometimes^ most notably in components -of -variance problems^ a statis- 
tician is trying to estimate a parameter he knows is positive, but in using 
standard unbiased estimation techniques he obtains an estimate which is 
negative. When this happens, the statistician would usually estimate the 
parameter by zero, since in so doing he is coming closer to the true param- 
eter value than the original estimate. 

Occasionally the statistician may have several unbiased estimates of 
the same parameter arising from independent sources. The statistician will 
want to combine the raw data i>ets andL treat them as one dara set; hov/ever, 
this may not always be possible, there may be nuisance parameters which 
vary from one source to another. Or the computational procedure may 
require too much memory to accommodate all the data at once. This may 
happen in using Rao's MDIQUE technique [8], where matrices the size of the 
data set are handled, or Henderson's third method with ^ large number 

of groups. (For a problem v;here either or both of these techniques are 
called for, see [ll])^ Finally, the statistician may not be able to com- ^ 
bine the observations due to not being supplied with the raw data. 

Xet us assume that from each source the statistician has nothing 
more than an unbiased estimate (v/hich can be positive or negative) of a 
common uriknown parameter value which is necessarily positive. We study 
alternative procedures for combining the estimates into a single number. 
The estimates are assumed to be identically distributed. (This does not 
cover the situation of- nuisance parameters varying over experiments, but. 



when caution is used, some features of the results reported here may still 
apply-) 

The main part of this paper is devoted to studying two simple technique 
for combining the estimates* A few other ^procedures are treated in a 
limited way in the last section. We study the performances of the two 
techniques when the distribution of the estimates is normal and when it is 
the weighted difference of two independent chi -squares with known degrees 
of freedom.^ The latter model is the correct one for one-way components of 
variance, and may be considered as a prototype for more complicated problems 
As the degrees of freedom vary, a range of distributional shapes is ob- 
tained. The normal distribution is, of course, the limiting case as both 
degrees of freedom become large. 

2. THE TWO ESTIMATORS AND THEIR MEAN 3QUAEE ERRORS 

I 

Let X. ,X^, .-mX be a set of independent random variables, each 
■^7 2 n 

member representing the estimate of" the unknown parameter \x based on 
one source* The X^ are assumed to be identically distributed with 
expectation [i = E(X^) and one nuisance parameter which is taken to be 
the standard deviation a = a(X^) . vie know that [i is greater than or 
equal to zero, although any X^ has a positive probability of being 
negative. 

Denote by xt the positive part of X^ , i.e., the random variable 
which equals X^ when X^ > 0 and which equals zero otherwise. When 
n = 1 , xt is the obvious estimator of [i . When n > 1 , one can 



either average the valuer and then take the positive part or take the 

positive parts before averaging* Denote .(X)*** , the estimator obtained by 

the first method, by (i , since it is the maximum-likelihood estimator under 

normality- We denote the second estimator (zxt)/n by \i , since it is the 

arithmetic mean of a set of quantities. These two estimators were con- 

.sidered by Sirotnik [10] in a mental testing application. 

We note that |i is not consistent, since it converges in probability 
< 

to E(xt) which is always greater than [i > i.e., E(Xp = 

P(X. > 0)E(X^|X^ > 0) > P(X. > 0)E(X. IX, ^ 0) + P(X. < 0)E(X. ix^ < O) = E(X.) 
One's first reaction to this fact is that [t has better properties and is 
thus the preferred estimator. This is certainly true when n is large, or 
even moderate. The reason for presenting this article is that, for small 
n , \i performs better over a reasonably large portion of the parameter 
space, assuming as we do that the X^ are either Gaussian or the weighted 
difference of two independent cihi-squares. ^ 

The comparison of the estimators* performances is based on mean square 
error (MSE), which is defined as the expected squared difference between 
the parameter and its finally estimated value, and which equals the vari- 
ance plus the square of the bias. It will be seen that formulas for the * 

2 

MSE contain a ^proportionality factor of a ; for this reason the compari- 
sons to be presented are in terms of MSE/a . 



Insert Table 1 about here 



Table 1 is a summary of the relative performance of \x and \x , as 
measured by MSE/a^ . When \i/a > l/2 , [I. is generally better, but 
the advantage becomes minimal as n increases, and the critical value of 
\x/a such that |I ^ is as good as \x is increasing in n and eventually 
exceeds l/2. The value of MSE/a for both ji and [I increases to l/n 
as \x/a approaches its theoretical upper bound"^ M . However, if we re- 
strict consideration to the region ki/a < K , for any K < M , the maximum 
MSE/a^ of 4I over this region is. less than that of (i when n = 2 or 3 
and, depending on ths distribution of , sometimes also when n ^ k 
or 5 • 

The relative advantage of \x over \x when \x/a > l/2 also depends 

on the distribution of the X, . In the next section some numerical 

1 ^ 

results are presented for the normal case and for the chi -square dif- 
ference with certain specified degrees of freedom. From these results we 
shall see in detail under what conditions \x is the preferred estimator. 

5. NUMERICAL CALCULATIONS AND GRAHIS 

When * n is large, |I is clearly a poor competitor to [i , since in 
this case [i will be close to |i and |I will be close to a number known 
to be larger than |i . Let us therefore see what happens for n ranging 
from 2 to 5, and to start with assume the to be normally distributed. 

The calculations are obtained from formulas (l^'.5) and iS-"^)* ^o*^ *hat 
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2 

for both \i and [i the MSB equals a times a function of the standard- 
ized mean m 8 [i/a (the reciprocal of the coefficient of variation). 

2 

For this reason we graph MSE/a as a function of m ; we use m rather 
than l/m = o/[i to display results for values of m close to zero, since 
the behavior as m «> is obvious* 



Insert Figures A and B about here 

To separate the curves, those for n = 2 .and k are shown in Figure 
A and for n = 5 and 5 in Figure B. 

It is seen that |i enjoys a healthy advantage over yl in the region 
1/2 < [i/a < 2 . When |i is much smaller than a the bias of ^I has a 
noticeable effect, and when [i- is much larger the chances of a negative 

X. are slight so that the values of both^ p and |jl will usually coincide • 

- - 2 

with X . Note that for the estimator X the value of MSE/a is the con- 
stant l/n . This value is approached by the curves for both \x and [x 
as m gets large, but the (i curve gets there faster* 
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Table 2 indicates the behavior of the MSB functions for larger values 

2 - 

of n . It is clear that for n as large as 9, the MSB/a of ^I is so 

- o 

much greater at m = 0 than the MSE/a"^ of (i is anywhere that one would 
want to avoid using |I for n > 9 even though it is better^ than \i for 
some values of m • 
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This gives the general picture vhen the are normal; for very 

small n there are certain advantages to \i , but as n increases these 

2 

are outweighed by a very high VISE/ a in the neighborhood of m^= 0 The 

next question is, do the small-sample advantages of \l persist when the 

X. are distributed other than normally? 
1 

In the one-way components of variance problem, the usual unbiased 

estimator- X of the main effect variance component has the distribution 

2 2 2 2 

of - OgXg , where and Xg are independent chi-square random 

variables with known degrees of freedom f and fg , respectively, and 

and 0^ are positive numbers related to thj unknown variance com- 
ponents and which satisfy the inequality E(x) = Q^f^ - ©^f^ > 0 • There 
are four cases to consider: low f^ and low fg , low f^^ and high f^ , 
high and low f ^ , and high f ^ and high • The low and high 
degrees of freedom have been chosen as 2 and 20* Other even values are 
easily treated using the fomulas of the next s^eciign* Odd-^numbered values 
are much more difficult mathematically, but by continuity we expect the 
characteristics to be similar to the adjacent even values* 



Insert Figures C and D about here 



2 

Figures C and D show, respectively, some MSE/a curves for f = 2 , 
fg = 20 and for f^ = 20 , f g = 2 • The curves for = f ^ = 2 have 
been examined and their appearance is similar to Figure C, with the \i 
curve having less upsweep near m = 0 . Some values were also calculated 
for f = fg = 20 , and they were reasonably close to the values for the 



normal case (see Table 5)« The conclusion my be drav^n that when is 
hi.gh thr; estimator \x is considerably better than \i for moderately 
high values of m , but that when is low the advantage is so meager 
that it does not seem worth the risk near m 0 • 
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To summarize> in problems where the can be considered to be a 

weighted difference of two independent chl-squares, we would recommend \i 
over (I if n is quite small, the degrees of freedom of the first chi- 
square is fairly large, and it is thought likely that m > 1/2 , i.e., that 
the mean ot is at least half its standard deviation. 

In a one-way variance components model where it is desired to combine 

estimates, from n independent sources, of the common variance components 

2 2 

(between) and a (within), this means we recommend that negative 

2 2 

estimates of be replaced by zero before averaging provided n is 

bo 

small, the number of levels k in the one-way classification is large, and 

the ratio a^/(var S^)^ is at least 1/2 . From [2, page 522, formula 

(5*7)]^ the latter condition in. the balanced case is equivalent to 

of/o^ > [/^ + /{/^ + (kr - 1 )/(kr - k))^]/r , where r is the number of 
0 w ^™ 

observations per level (within one source) and / = (2k - 5) ^ • A more 

2 2 

simply calculated quantity which exceeds the above lower bound for ^^/^^ ^ 

provided k > 5 and r > 2 , is 2//r . If k = 26 and r = 10 , for 

- 2 2 

example, |I is preferable to (1 when > cr^/55 (and n is small). 



If. DERIVA^iJlON OP POUMUIAS FOR MEAN SQUARE ERROR 

We now indicate how the. values for the curves discussed in Section 2 
were calculated*^ 

2 

As before, denote by \i and o the mean and variance, respectively, 
of the , and for any U denote = Tnftx{U,0) . We wish to derive 
formulas for the MSE of {x » and {i & (ZX^/n • These can be obtained 
ft^om expressions for the mean and variance of X and of X^^ , since the 
MSE of any estimator of \i with.imeah v and variance r ' is given by 
T + (v - |x) . 

2 

If F is the c.d»f. of X^^ , the mean v and variance r of ^ 
are given by 

V « E(xJ) = J xdiM , (If.l) 



nr^ + «= E{(Xp^) = j x^dF(x) . (lf.2) 



For the normal case, let z = (x - ji)/a ; then 



j xdF(x) = J iv-* az)d*(z) , 
0 -ji/a 

J x^dF(x) = J iv^ * 2jiaz + a^z^)a*(z) , 
0 -u/<y 



where ^ is the standard normal c«d«f« The integration is straightforward^ 
using integration by parts on z d»(z) . Hence Vg ^ atn<)(m) ^(m)] 

and nx = a , where ♦(m) = (2k) ^ exp(*ni /2) and = 

2 2 
m ♦(m)fl - *(m)) - w#(m)(2»(m) - 1) + ♦(m) • U(m)) . Finally the ^BE 

for \x is 

where 6^ = ♦(tn) - in{l - ♦(m)) . Since X has a normal distribution, the 
MSE for fi way be derived similarly; its value is 

The distribution of a weighted difference of two independent chi- 
squares with even degrees of freedom can be represented a5^, a finite mixture 
of positive and negative chi^squares. This result has been derived 
Ariously by Box [l]^ Mantel and Pa&ternack [6], and Jayachandran and Barr 
[5]* Using the notation of Section 5 with p « f,/2 and q « f^2 as 
integers the density oj X r ©^X^ - ©^Xg can be written as 



^ 3=1 \ ' ^ / (1 + r )?** J ^ 



= 2ijC^(r;p,q)g^(x) + ^dj^(r;p,q)hj^(x) , 

say, where r = Oo/®i ' S.(x) is the density of 0^ times a chi- 
square with 2j degrees of freedom and hj^(^) is the density of (-Og) 
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times a chi-square with 2k degrees of freedom. Note that the g^(x) 

are nonzero only when x is positive and the h^(x) are nonzero only 

when X is negative. Since (1^.1) and (1^.2) involve integration over only 

positive values of x , we need consider only the terms containing g^(x) . 

The e:^ectation of xt is therefore the weighted sum of expectations of 

0 random variables with the weights given by the coefficients of the 

1 2j 

g.(x) in (k-,^), and the second moment of xt is a similar weighted sum of 

2 

1^2j 



V 2 2 2 

second moments of these Q,)Q . -Using the fact that a = kO^(^ + r q) , 



ECxt) = Z c (r;p,q)(2j0 ) 
0=1 ^ 



2 i P 
= a(p + r q) 2 Z jc .(r;p,q) 



and 



E{(xt)^) = [a^ + r^q)] Z j(j + l)c.(r;p,q) • 
^ j=l ^ 

2 

These formulas were used to compute MSE/a for the estimator \i as 

2 

a function of r . The quantity MSE/a was then plotted as a function- of 
m by obtaining m from the monotonically decreasing relation m = 
(p " rq)(p ^ r\y'^ which follows from [i = 20 ^^^(p - rq) and = ' 

U0^(p + r^q) . Since (i > 0 , the maximum 'value of r allowed is p/q ; 

1 i 

also r > 0 implies that m < p^ = {tj2)^ . 

2 2 

Note that the distribution of X has the representation Q^X^ - O^Xj^ , 

2 2 

where 0^ = Q^/n , 0^ = 02/n , and and X^ have nf and nfg 

2 X 
degrees of freedom, respectively. Thus MSE/a may be calculated for (i 

in the same manner as 'for (I , with the appropriate substitutions. 
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5* OTHER POSSIBLE ESTIMATORS 

In this section we look at a few other naturally suggested estimators 
of simple form. Motivated by the results oust stated, the author decided 
to seek a simply calculated compromise between (1 and \L , one that does 
better than p for larger values of m = [i/a but does not have the 
characteristically high MSB that (I has for values of m near zero* It 
is appealing to try to use the sample informati'^n to get some idea of m 
and then choose ft or |I accordingly. An easy rule to use would be 
based on the proportion of which are negative. If this proportion 

is high, it indicates a greater likelihood of low values of [i/a , and we 
would perhaps be predisposed to use the rule that performs best for these 
low values. 

2 2 ' 

For brevity vje restrict ourselves to the model 9^X^ - OgX^ where 
both X^ and X^ have just two degrees of freedom- This turns out to 
be the easiest case mathematically. From C^*?)^ 

f^(x) = [Q^g^M Q^h^(x)]/(Q^ ^ ©g) . 

2 2 

Here is distributed as a mixture of a 0-|^X and a -OgX random 

variable, .each with 2 degrees of freedom, with weights proportional to 

and respectively. Regard the sampling as coming from an urn 

with "positive" and "negative" balls in this proportion. For a small 
sample it is easy to wite the probabilities of observing J negative 
balls ( J taking values from 0 to n ) and to compute the mean square 
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error for any rule which chooses \i or \i according to the value of J . 
Letting Y stand for the resulting estimator,. 

MSE = E(Y - [if = f Pr{J = j}E{(Y - = j} • 

Taking as an example n =^ 3 , by symmetry we can wite 

E{(Y - n/l J = 1) = E{(y - II fh^ < 0, Xg > 0, > 0) . 

When < 0, Xg > 0 , and X^ > 0 we have X^ = 0 , Xg = Xg , and 

= , so that here |I = (Xg + X^)/5 , where Xg and X^ have the 

2 

conditional distribution of two independent O^X random variables, each 
with 2 degrees of freedom. Hence E{[I|j = l) = ^-J^ and Var{|I|j = l) = 
80^/9 . The formula for E{(|I - J = l) follows directly. The con- 
ditional distribution of X > given that J = 1 , is that of (©3_/5)\ - 

pop . 

(02/5 )X^^ , v;here and X^^ are independent chi-squares with 4 and 2 

degrees of freedom, respectively. Thus the first and second moments of \i 

may be calculated along similar lines to those indicated in Section k. The 

J = 2 case is similar, and the J = 0 and J = 3 cases are trivial, 
o p 

If or have an even number of degrees of freedom more than 

two, the same general principles may be used, but the distributions, given 
v/hether positive or negative, are no longer pure chi-squares but mixtures 
of Chi -squares. The computations for \i then become rather involved. 

VJhen n = 3 there are just k possible rules of the type described 
above> based on choice of (i or il when J = 1 and when J = 2 , two of 
which are simply jl and \i . (When J = 0 or 5 , |1 and \i yield the 
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same value.) Let tl^ be the estimator obtained by choosing \x when 
J = 1 and \i when J = 2 . (This is the rule based on the heuristic 
likelihood argument of the first paragraph.) Let (1^ be the estimator 
obtained by choosing \x when J = 2 and (x when J = 1 . 



Insert Table h about here 



For selected values of r ^ ' Table h gives the corresponding 

value of m = ti/a and MSE/a for the rules , w 7 ^^-^ and , 
in the case ^i"^ ^ * Surprisingly, the rule which chooses ^ 

or \JL according to the relative likelihood of low or high values of m 
performs badly, and the rule ilg which reverses the choice does relatively 
well, tig has a lower MSE than (1 for all m greater than about .28 
and a lower MSE than either fi or (I for m between .28 and .55- At 
m = 0 , where vl loses to (i by .083, the compromise estimator jlg is 
worse than (1 by .05I. (Ig is thus a much more acceptable alternative to 
than is |i . 

We present a possible explanation for the fact that the MSE for 
is usually lower than for • When two out of three of the are 
negative, we know that these X. are less than |i and hence that' X is 
probably an underestimate of [i . The estimator = X^ compensates for 
this to some extent by frequently yielding zero in these cases. This is 
good if [I is close to zero. But for larger [i (i-e., [i > .28a ), the 
estimator \i which is strictly positive in these cases seems to perform 
better. 
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When only one X^^ is negative, on the other hand, X will usually 
be positive and as such will usually be closer to [i than will the biased 
iT • This holds unless [i/o is quite high (> -55), where now even one 
negative X^ is unlikely and when it occurs it is evidence that X will 
underestimate p. , so again the estimate should be liaised. 

It should be pointed out that this article is written, from the 
xrequentist point of view, according to which expectations are based on 
repeated sampling with the same parameter values* This viewpoint has 
been challenged by many statisticians (see, e.g., [?]), and alternative 
cx^iteria of performance might conceivably yield different results. 

Also included in Table h are results for {I , the positive part of 

the sample median. Its mean and variance are easily calculated in the 

' 2 

f ^ = f 2 = 2 case since here G^X^ is exponentially distributed, and 
using the no-memory property the order statistics are directly expressed 
as convolutions of exponential random variables; see, e.g., [3^ P* 55r 
Frop. 3]. [I performs better than its competitors when m is small and 
worse when m is large. In the case of large m the distribution of 
X^ is markedly skewed, and we would thus expect the sample median to be 
centered around the population median, but not the population mean. As 
m 0 the distribution of X^^ approaches the double exponential, for the 
case f ^ = f 2 = 2 , for .v;hich the sample median is the maximum-likelihood 
estimator of the center of symmetry and is known to have good properties. 
This feature would not be expected to be present throughout all the dis- 
tributions for X- studied here. 
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6. CONCLUSION 

We have compared two simple rules for the problem of combining 
independent, identically distributed, unbiased estimates of a parameter 
value known to be positive when the estimates may be negative. The first 
rule ^ (the positive part of the average of the estimates) is consistent 
and the second rule |I (the average of:the positive parts) is inconsistent. 
In large samples there is of course no difficulty in choosing between tho 
two. We have seen that when n Is very small the relative performance 
depends both on the ratio \i/o (when yi/c > 1/2 , \i is^ generally 
better; otherwise ft is better) and on the underlying distribution. A 
rule of thumb for when to use |I in practice is suggested in the last two 
paragraphs of Section 3; hov;ever, this requires a prior idea of \i/a and 
of the underlying family of distributions. A good practical rule is not 
obvious for situations where no such prior knowledge exists. 

For the case of the chi-square difference with (2,2) degrees of free- 
dom and n = 5 , the positive part of the sample median performs very 
well when ti/a is small, but badly when tx/cr is large. The estimator 
llg (choose (I if two of the independent unbiased estimates are negative) 
performs better than the others in the intermediate region and also 
reasonably well for large (i/cr • 

All estimators studied here have a positive probability of being 
zero. It would be desirable to have a simply calculated estimator with 
the property of always being (strictly) positive, or at least of being 
positive whenever one or more of the is positive. To date no 

reasonable method of obtaining such an estimator seemt to be available. 
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''ror the normal case M = +« , and for the weighted difference of two 
chi --squares M is the square root of half the degrees of freedom of the 
positive chi-square (see Section 1*-, second last paragraph). 
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1. SUMMARY OF PERFORMANCE OF |I RELATi\^ TO [I 



n 



2,5 



6-8 



> 9 



a 2 
a 2 



estimator 
with lower 
maximum^ 



|I better 



|j. worse 



|I better |I better very close 
|I worse |I 'worse \i much worse 



depends on 
distribution 
of X. 



^"Maximum" refers to maximum MSE/a over any region 
of the form 0 < \x/a <K <U where M is the largest 
possible value of \x/a . 
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I^. VALUES OF MSE/o^ FOR P , il , , AND p WHEN 







n = 5 AND 


^1" 










m 






MSE/o^ 
















0 


1.000 


.555 


.555 


.555 


.555 


.589 


0.2 


0.7814 


.527 


.515 


.519 


.521 


.561 


O.k 


0.557 


.289 


.275 


.288 


.276 


.290 


0.5 




.261 


.259 


.270 


.250 


.255 


0.6 


OM 


.2514 


.2I48 


.255 


.228 


.220 


0.7 


0.2146 


.209 


.2142 


.259 


.212 


.191* 


0.8* 


0.156 


.189 


.2I4O 


.228 


.201 


.175 


1.0 


0 


.167 


.250 


.219 


.196 


.160 
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